Overview of the NTCIR-12 QA Lab-2 Task
نویسندگان
چکیده
The NTCIR-12 QA Lab-2 task aims at the real-world complex Question Answering (QA) technologies using Japanese university entrance exams and their English translation on the subject of “World history”. The exam questions are roughly divided into multiple-choice and free-description styles, and have various question formats, which are essay, factoid, slot-filling, true-or-false and so on. We conducted three phases of formal runs, and collaborated on Phase-2 Japanese subtask with the Todai Robot Project. Twelve teams submitted 148 runs in total. We describe the used data, the hierarchy of question formats, formal run results, and comparison between human marks and automatic evaluation scores for essay questions. Categories and Subject Descriptions H.3.4 [INFORMATION STORAGE AND RETRIEVAL]: Systems and Software Performance evaluation (efficiency and effectiveness), Question-answering (fact retrieval) systems.
منابع مشابه
Forst: Question Answering System for Second-stage Examinations at NTCIR-12 QA Lab-2 Task
Japanese university entrance exams have two stages: The National Center Test (multiple choice-type questions) and second-stage examinations (complex questions including terms and essays). We participated in all phases of NTCIR-11 QA Lab-1 and NTCIR-12 QA Lab-2 task’s Japanese subtask and our system answered all of the questions. At QA Lab-2 task, we focused on term and essay questions in the se...
متن کاملOverview of the NTCIR-11 QA-Lab Task
This paper describes an overview of the first QA Lab (Question Answering Lab for Entrance Exam) task at NTCIR 11. The goal of the QA lab is to provide a module-based platform for advanced question answering systems and comparative evaluation for solving real-world university entrance exam questions. In this task, “world history” questions are selected from The National Center Test for Universit...
متن کاملCMUQA: Multiple-Choice Question Answering at NTCIR-12 QA Lab-2 Task
The first version of the UIMA-based modular automatic question answering (QA) system was developed for NTCIR11 QA Lab task[5]. The system answers multiple-choice English questions for the Japanese university entrance examinations on the subject of world history. We made improvements in the current system by adding components focused towards Source Expansion and better Semantic Understanding of ...
متن کاملKSU Team's Multiple Choice QA System at the NTCIR-12 QA Lab-2 Task
This paper describes the systems and results of the team KSU for QA Lab-2 task in NTCIR-12. In each phase of the task, we developed three automatic answering systems for world history questions in the National Center Test for University Admissions. In order for QA systems using document retrieval to answer questions correctly, it is important to estimate exact question types, and to utilize kno...
متن کاملIMTKU Question Answering System for World History Exams at NTCIR-12 QA Lab2
In this paper, we describe the IMTKU (Information Management at Tamkang University) question answering system for Japanese university entrance exams at NTCIR-12 QA Lab-2. We proposed a question answering system that integrates natural language processing and machine learning techniques for Japanese university entrance exams at NTCIR-12 QA Lab-2. In phase-1, we submitted 6 End-to-End QA runs res...
متن کامل